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ABSTRACT ' , 

This research exaaines the interaction between 
college students* control orientation and a discrepancy score of GPA 
minus expected grade- in cputse/on the dependent measure of 13 
student rating items. It was hypothesized that students with an 
external locus of control who also showed a ^discrepancy between 
expected ^and actual gra&es would distort ratings in. the jdirection of 
the discrepancy. Regressions w^re perfoxted on daiba froa 200 Ss» In 
four regressions,^ th^ interaction tera was significant; thus the 
hypothesis was partially supported. As the§e four itets had be^n 
shown to l?e particularly iaportant, the influence of locus of control 
.is interpreted a^ a reduction in, validity of studept rating 
instrunents, (Author) ' ' , m * ' 
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Introduction and Objectives 



Student ratings of instruction continue to grow in popularity *and use* 
The • i^esearch literature, comprised gener^^y of tfcale development and vatlidity 
studies, is largely supportive. of st\;ident re^tings, and this support surely has 
heiped to increase ratings' popularity* fiowever, two major problems occur re- 
peaVedly in research. on student ratings of instruction. Flrst^ many research-*- 
ers imply that positive outcomes, with a particular scale at a particular 
school shJkld generalise to all scales and schools. There is ample evidence 
to stxggest that scale properties do not generalize so easily (Owen, 1976; 
Doyle and Whitely, 1974) t and more espepially, that even students ^of different 
major areas perceive instructor competencies differently (Slater add Owen^, 1974)* 
Second, many studies have used univariate statistical analyses, sometimes re«- 



suiting in ^reat loss of data and perhaps a distortion of the results* 

The construct pf locus of control has been shown to relate fo a large 
number of behaviors: * academic achievement, political activism, responsiveness 
^o external feedback, peychopathology, - ability to del^y gratification, and'vo 
^n (Lef court, 1976)* The literature is spanfe, bowever^* on the relatioiiship 
" between ^ocus of dontrol and students* perception of teacher competence* A 
single study by PlsAders, Norrieon, and BMde (l968) showed tbat 6th^grade 
^etudente with an exifmi^ loous oX dpqtrol showed more decline in attitude 
■ toward iaaoh.ra ovar th. school ye£> than did th.lr Int.pn^ paara. Thay 
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hypothesized that the external youngsters* attitudes reflected their depen- 
deuce on external sources of feedback (e«g*| the absence of teacher praise)* 

Such a relationship between <:ontrol orientation fuid attitudes would ' ^ 
suggest an interesfing individulil difference in perception and evaluationt 
but xt would also be damaging to the validity^ of a rating instrument, as it 
points to a systematic source of extraneous variance/ 

Arother variable which has been shown to influence student ratings* is 
ths expected gradj^ in a course (jnfeaver, I960)* 'Yet, a related- variable is 
^ typically even more striking in its influence on ratings: the discrepancy 
between expected and actual grades (Bausell and Magoon, 1972; Kennedy, 197^t 
Lollif 1976)* In brief, these students who expect a grade hi^er than^theirr-^ 



actual grade tend to be "generous** in ratings, ^ 
Because discrepancies in actual and expected grades represent a 8#urce 
of dlssonemt and external feedback,' it mi^t be predicted that students with ^ 
^ an external locus of control would be most influenced by discrepancies, and. « 

y ■ 

int^nal students, least influenced* Thus, if this hypothesis holds trae, 
an interaction between Tlocub of control and grade discrepancy would be seen* 

• V 

jThe present research was designed test thisl interaction. . . 

MethodlB r 

One hundred ninety-six college students in geography and educflltion were 
administered two instruments- in the last week of classes, and prior to the 
award of^nal grades | The instruments were, first, the Mowickt-Dukm Locus 
of Control Scale for^Aclults (1974)9 cLnd an instructional rating form whose 
development is described eldewhere (Lolli and Owen, ^976)* The instructional 
rating form consists of several bits of demografdiic data (ive*, semester stand 
ing, expected grade, etc*), 13 high inference .evaluative it^ms (i*e*. 



demonstrates overall organization; almost never* ••alnooit always)/ and* ^1 low 

f * / ^ . - ' 

inference itens in a ch,#cklist format (l^e*,., spbkie^ih a 'ii)onotone)('see^ppen(thc 

a).. The two data sources were administered in a rimdom order to studentSi to^ 

control for a potential sequence effects in testing* A discrepancy scpre was 

* r , * 

created for e^^h studexit by subtracting the self-reported grade point average 
(GPA) ^rom the expected grade in l^e course* 

Usin^stepwise multiple rfygre98i9n techBiquesi . the data were analysed 
in 'Uiis fashion: th;e two main ^ffects-^locus of . control scorsi and the die- 
crepancy score^'^ere forced into the regression equation*' The interaction 
between these Xyp ef fei^^ was th^n entered and *tested.for incremental validity 



as 



follows: 



K ' Pi (R^ - Rg) • / ' (a^ - «2^' 



(1^ . R^) / On - m^ - 1). 



(Kerlinger and Pedhazur, 1973» P* 178) 

' ' " »■ 

the regressions were repeated 13 times) in each catse iising one of the 13 high 
inference rating items as the dependent variable^ 

Table 1 depicts the means and standard deviations on all variables* \ On 
three of the 13 multipl^e regression analyaesi '^he interlu^tion between Ibcus ^ 
6f dont;x>l and grade discrepancy slj^wed significant Incremental, validity (see 
Tabler 2)* Two additional interaction terms approached significance*. Althou;^ 
the Rs were low^ (.1^ to *24)i^ the occasional signlflpance of the interaction 
term suggests that some systematic variation In student ratings can be attri- 
buted to .the personality characteristics of locus of control* jUfi inspection 
of the ^ three sij^lflcant^interaction terns reveals the pattern lm>othesisedi* 
External^ students tdho expected a grade hij^ef than their OPA f reward** the 
instructor; externflds who expects a grAde lower than their OPi^iometimeB respond 



with a lowQr rat:^ng of the instructor. For illustrative purposesi Figure 1 
shows the interaction between control Orientation and grade discrtpancy for 
evaluative, item 5» \ ^ 

The results take on clearer nature when the dependent otMeures %are 
examined more carefully. In prior principal coiM6o^^nts studies of the 13 
high inference jkems (here used as criteria), t^fe principal components emerged 
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and.we-]|e named "teaching ef fectiveness** and "interaction with students*" ^As 
ityhappens, the three criterion itemis Bignifi>€uitly predicted by the intei^ 
action term in the present research are the same three items which sttoWed the - 
hi^est component loadings An the earlier studies;' The component loa4ing8 of 
•80 to . 90 show that these three items weight heavily in the overall construct 
validity of the scale. It can be inferred in^ turn' that the present xegwssion 
results are not merely circiunstantial; they represent swall^but eystematio 
.sources of "noise" in the rating of teachers. 

The practical significance of the present zjBsultB is not .yet clear* 
Obviously, replications must be carriecf out to determine the general i aab il i ty ' 
of the resultp. If* the results prove general ijable, should we disco\int exter^ 
nal students' ratings of their teachers? It could be argued that because the 
eiffect occui^ only opcasionally, and because it is not a mamslve source of 
variation, We should live with it as error variation. On the other .hand, 
teachers whose livelihood depends partially on student ratings, migjit insist 



that the data be "cleansed*** Co^idfer. further the possible rating distortions 
that .mi^t occUr in a class i*iere*the inatrutetor were overly "generous,", or 



where the instructor is a "hard" grader*. ' ' • t 

'^^^-^^CuTTentJy uaderway are Analyses intended^^to comfiine'^everal nagging' 
sources of extraneous variation in ratings* Of interest h^he is irtiethtfr the' 
sources are additive, i>r (^verlapping and thus less damaging to the construct 
validity of rating instruments* ^ ^ ^ - 
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Figure 1 



Plot of Interactij>fi Term for Ei^aluative Iten Five 
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To illuiBtrate the interaction regression ^lane for iten 5, students with locus 
of control Scores one s.d. above the oei^n (l2)i and one s«d« below the meah ^4) 
were chosen. They are labeled, respectively, as "external" and /'internal^ / 

"Ratings on all items range from «ero (low) to five (hi^).* 

Discrepancy scores were coded from -7 (low OPA expecting a higji grade) to 48 
(hij^ CPA expecting a low grade). 
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' V^ri^les. 1 throu^ 13 represent ratings Items; tmir stems-n 
are ^ven in Appendix A* LOC » Nowioki-^Doke Locus^ of ControQ 
Score; DISC m Disorepaacy between ejtpected^grade and current 
grade point alverage; XPROD « Interactioi^^^tenn of LOC and DISC. 
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Table 2 ^ \ ^ ^ . 

Summary of R^gressipn Outcorfes for Prediction ^ . 

^ of Thirteen Evaluative l-teniB (N . 196) 
^ t * 

1 '2 * 2 ' ^ InoT«Mntal 
/ Item Predictors . R R Validity tWa^ 

i LOG ■ ^roS4 Toot , ' 

• • , . ■ DISC , .096- - .009. .• - • 

•-^ XPROD - .120 ■ ^014 1.01 (n.B.O • ' 

-2 "* LOC .132. -.017^ ' . * 

• ' DISC .133 ' .OlS / ■ , • 

XPrfOD' .192 .037 ^' 3.71 (n.B.) 

- 3 , ' LOC - '* ;1^ .028. " ' ' • ' J 

lisc • .170' ' .029 . , - 

XPROD .170 .-.029 ' ^ 0.04 (n. 8.). ^ 

( ^ 4 LOC , ' .055 • .003 ' ' ' . 

^ . . • DISC .059 . .003 



XPROD .152" *023 3.80 (n.B.) 



^ V .. . 

XPROB .185 . .^ .034 5.51 (P4<|05) 
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DISC ' .080 - ,0P6 



LOC ■ .053' ■ .003 * ^ ^ \ 

DISO ,057 N ' .Cfe3 

xhtOD .058 /-.OOB 0';d2'(n.B.^ 

LOC ■ "' ' .080 .006 V ^' ' 

DISC .,081 ' .007 . 

XPROD, .084 .007 ,0.09 (n.B*) 



8 -LOC .010 .000 

DISC .021 .oob 



' XPROD .036 _ .001 #0.15 (n.B.) 
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■ XPROD ,.144 .021 ' t).04 (n.B.) '. . 

10 LQC .011 .000 
DISC- .138 • .019 ^ 

XPROt) ,.138 .019 0.09 (n.B.) *\ 

11 LOC .038 .001'"' ' ^ '■■ ■ 
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•XPROD .237 ..05i 5.69 (p«,< .65) 

IJ ' LO^ , .042 ^*002. 

. • ' DISC .053 . .003- ' 

XPROD .229 .052 \ 8.50 (i>'< .01) 



DISC, .043 .002 
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For iteji straSf see Appendix A. , ^ 

2 • • • . , . 

JfiC m Vowicki'-Duka Locue of Control Score; DISC »- Discrepancy between 

I etpected ^prade and current grade point avef^ige'; XPROD « Interaction 

^ * term of LOC^and DISC. j ^ 
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